Skip to content

[RFC] Broaden support for profiling generated code#341

Merged
Erotemic merged 1 commit intopyutils:mainfrom
matthiasdiener:gen-code
May 16, 2025
Merged

[RFC] Broaden support for profiling generated code#341
Erotemic merged 1 commit intopyutils:mainfrom
matthiasdiener:gen-code

Conversation

@matthiasdiener
Copy link
Contributor

Currently, packages that generate Python code aren't easy to profile with the line_profiler, unless doing some contortions to adjust the file name to something that triggers line_profiler to look into the linecache instead of the file system. This PR slightly broadens the existing support for ipython to support more code generators.

@Erotemic
Copy link
Member

The code changes seems reasonable, but I would like to know more about the motivation. What sort of non-ipython system would uses the <generated tag and would benefit from this?

@inducer
Copy link

inducer commented May 16, 2025

I can think of a few precedents for generated code:

Perhaps the more salient point is that, IMO, it is not safe to assume that co_filename corresponds to an "actually accessible" file name. Instead, the default/only safe way to resolve these filenames is via linecache. I learned this the hard way when implementing https://github.com/inducer/pudb.

@Erotemic
Copy link
Member

This makes some sense, but it would help to see an explicit example (i.e. a new test) that demonstrates how line-profiler is missing this case and how this patch fixes it.

I don't understand how dataclasses could be used in a way that would cause trouble. I'd be interested in seeing that example.

We are already using linecache to get the lines (although we clear it if the file exists, which may be the wrong thing to do unless perhaps we are in a reloading IPython context), so I think this patch would just allow code with a filename starting with "<generated" to pass the condition to access its lines via linecache, I would just like a test-case that covers this before I accept the patch.

@matthiasdiener
Copy link
Contributor Author

matthiasdiener commented May 16, 2025

The specific use case I had in mind was this code generator here: https://github.com/inducer/pytools/blob/7638eacdb2ef690504bade0afc63fc11330b5e85/pytools/py_codegen.py.
In particular, https://github.com/inducer/pytools/blob/7638eacdb2ef690504bade0afc63fc11330b5e85/pytools/py_codegen.py#L66-L68 creates the name for the generated code.
Without this PR, we would have to change the name created there to something like <ipython-input- ... in order to support profiling with line_profiler.

@Erotemic
Copy link
Member

Is it possible to make a MWE that we can use as a test? I don't see an example of how to use PythonFunctionGenerator in a way that makes it obvious how it interacts with line-profiler.

@matthiasdiener
Copy link
Contributor Author

matthiasdiener commented May 16, 2025

pytools.py_codegen.py isn't yet using linecache, this is being done in inducer/pytools#295 (see e.g. the heuristics needed to support line-profiler: https://github.com/inducer/pytools/pull/295/files#diff-4e20ea316836b98bed526a3a92acb5c9f0aede15dfea15490f6994dcbc21b562R49-R55)
, so I've created an MWE that simulates the intended support, and can be run with the current main branch of pytools:

from pytools.py_codegen import PythonFunctionGenerator
import linecache

cg = PythonFunctionGenerator("test_fn", args=(),
                             decorators=["from line_profiler import profile", "@profile"])
cg("return 42")

fn = cg.get_function()

# Mimic https://github.com/inducer/pytools/pull/295
linecache.cache["<generated code for 'test_fn'>"] = (None, None, ["from line_profiler import profile", "@profile", "def test_fn()", "  return 42"], None)

fn()

Without this PR (#341):

$ export LINE_PROFILE=1
$ python t.py
Timer unit: 1e-09 s

  0.00 seconds - <generated code for 'test_fn'>:2 - test_fn
Wrote profile results to profile_output.txt
Wrote profile results to profile_output_2025-05-16T171619.txt
Wrote profile results to profile_output.lprof
To view details run:
python -m line_profiler -rtmz profile_output.lprof

$ cat profile_output.txt
Timer unit: 1e-09 s

Total time: 2e-06 s

Could not find file <generated code for 'test_fn'>
Are you sure you are running this program from the same directory
that you ran the profiler from?
Continuing without the function's contents.

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     2
     3
     4         1       2000.0   2000.0    100.0

  0.00 seconds - <generated code for 'test_fn'>:2 - test_fn

With this PR (#341):

$ export LINE_PROFILE=1
$ python t.py
Timer unit: 1e-09 s

  0.00 seconds - <generated code for 'test_fn'>:2 - test_fn
Wrote profile results to profile_output.txt
Wrote profile results to profile_output_2025-05-16T171545.txt
Wrote profile results to profile_output.lprof
To view details run:
python -m line_profiler -rtmz profile_output.lprof

$ cat profile_output.txt
Timer unit: 1e-09 s

Total time: 2e-06 s
File: <generated code for 'test_fn'>
Function: test_fn at line 2

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     2                                           @profile
     3                                           def test_fn()
     4         1       2000.0   2000.0    100.0    return 42

  0.00 seconds - <generated code for 'test_fn'>:2 - test_fn

@codecov
Copy link

codecov bot commented May 16, 2025

Codecov Report

Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 63.78%. Comparing base (7988790) to head (771d0b7).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
line_profiler/line_profiler.py 50.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #341   +/-   ##
=======================================
  Coverage   63.78%   63.78%           
=======================================
  Files          13       13           
  Lines        1041     1041           
  Branches      228      228           
=======================================
  Hits          664      664           
  Misses        316      316           
  Partials       61       61           
Files with missing lines Coverage Δ
line_profiler/line_profiler.py 66.66% <50.00%> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ea11bb...771d0b7. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Erotemic
Copy link
Member

Perfect, that's enough for me to at least accept the PR. It would be good you could do an additional PR where you add this as a test. Otherwise, I'll get to it eventually. It would involve putting that snippet in a some test/*.py file and adding pytools as an optional dependency to requirements/optional.txt. (Eventually we should move to pure pypyroject.toml).

As an aside, I was browsing pytools and was amused to see many of the same ideas in ubelt and other stdlib extension libraries (e.g. boltons, strif, @jaraco 's tools, toolz, etc..) . I'm interested in exploring the question if there is a subset that should be added to the stdlib.

@Erotemic Erotemic merged commit e35b6cf into pyutils:main May 16, 2025
35 of 36 checks passed
@matthiasdiener matthiasdiener deleted the gen-code branch May 16, 2025 23:59
matthiasdiener added a commit to matthiasdiener/pytools that referenced this pull request May 19, 2025
inducer pushed a commit to matthiasdiener/pytools that referenced this pull request May 21, 2025
inducer added a commit to matthiasdiener/pytools that referenced this pull request May 27, 2025
use linecache directly

only adjust filename if line_profiler loaded

improve args

fix

maintain compatibility with pudb

broaden line_profiler usage check

more type annotations

pyright ignores

rename generated functions to avoid linecache warnings

remove special pudb handling

make sure names are unique

fix bpr

expand to support PythonCodeGenerator, refactor

fixes

remove special line_profiler handling

Relevant PR (pyutils/line_profiler#341)
has been merged

Rework for linecache compatibility, improve tests

Co-authored-by: Andreas Kloeckner <inform@tiker.net>
inducer added a commit to inducer/pytools that referenced this pull request May 27, 2025
use linecache directly

only adjust filename if line_profiler loaded

improve args

fix

maintain compatibility with pudb

broaden line_profiler usage check

more type annotations

pyright ignores

rename generated functions to avoid linecache warnings

remove special pudb handling

make sure names are unique

fix bpr

expand to support PythonCodeGenerator, refactor

fixes

remove special line_profiler handling

Relevant PR (pyutils/line_profiler#341)
has been merged

Rework for linecache compatibility, improve tests

Co-authored-by: Andreas Kloeckner <inform@tiker.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants